Texture Capture Stylus and Method

ABSTRACT

Embodiments disclosed herein relate generally to a stylus for texture capture. The stylus includes an image sensing and capture device to permit a surface to be scanned using the stylus and an image may be stored and displayed on an electronic device to represent the texture of the scanned surface.

TECHNICAL FIELD

Embodiments disclosed herein relate generally to styli that may includeadditional components for sensing, recording and transmitting an imageof the texture of a surface.

BACKGROUND

In the visual arts, texture is the perceived surface quality of an imageor work of art. It is an element of two-dimensional andthree-dimensional design and is distinguished by its perceived visualand physical properties. Use of texture, along with other elements ofdesign, can convey a variety of messages and emotions. For example, useof rough surfaces can be visually active, while smooth surfaces can bevisually restful. Physical texture, also known as actual texture ortactile texture, are the actual variations upon a surface. Physicaltexture has a physical quality that is a combination of how the surfacelooks, and how it feels to the touch. For example, in a painting,physical texture is based on the paint, and its application, or theaddition of materials such as ribbon, metal, wood, lace, leather andsand either on the surface of, or mixed with, the paint.

Visual texture is the illusion of having physical texture. Everymaterial and every support surface has its own physical texture thatneeds to be taken into consideration before creating a visual imagewhich is meant to emulate that physical texture. For example, materialssuch as canvas and watercolor paper are considerably rougher than, forexample, photo-quality computer paper and it may be difficult to createa flat, smooth texture from an image shown on such materials.Photography, drawings and paintings use visual texture both to portraytheir subject matter realistically and with interpretation. Visualtexture in those media is generally created by the repetition of shapesand lines. Simulated texture creates the visual effect of texturewithout actually adding texture. For instance, texture may be created tolook like something other than paint on a flat surface. Alternatively,an image may create the illusion of ripples through the repetition oflines.

Today, the term stylus generally refers to an input tool used withPersonal Digital Assistants (PDAs), graphics tablets, Tablet PersonalComputers (PCs), and Ultra Mobile Personal Computers (UMPCs). Modern daystyli generally take the shape of a writing instrument, such as a pen,and are made to comfortably fit in the grip of a user's hand. Thesestyli can be found in all different sizes and shapes. Some styli mayextend and contract into small, pen-like cylinders, which are easy tostore. Other types of styli may be larger or contain additional devicessuch as microphones.

In electronic applications, the user operates a touchscreen with astylus, rather than using a finger in order to avoid getting the naturaloil from the user's hands on the screen. Use of a stylus also improvesthe precision of the touch input, allowing use of smaller user interfaceelements such as icons. Styli may also be used for handwriting ordrawing on a touch-sensitive surface such as a computer tablet screenand may assist a user to accurately navigate through menus, sendmessages etc. As electronic devices have evolved, they have been usedmore frequently by engineers, artists and other creative individuals forartistic drawings, image creation, graphic design and other uses.

When a user of an electronic device, such as a graphics tablet or a PC,attempts to create an image which includes visual texture, that user maybe limited in the type of physical texture he or she can simulate. Thus,it is generally left to the creativeness and efforts of the user todetermine the methods and types of visual texture which may be provided.Accordingly, an improved device, method and/or system for capturing,storing, and providing visual texturing information from an image of asensed surface to an electronic device may be useful in permitting theuser to create an image which emulates the texture of the surface thatwas sensed.

SUMMARY

A stylus is disclosed which includes an apparatus and system fordetecting, storing, and simulating the physical attributes of a texturedsurface. The stylus may include an image sensor contained within orattached to the stylus which senses the physical attributes of a surfaceover which the stylus is passed. The stylus may include a lighttransmissive tip through which the image sensor can detect the textureof the surface over which the stylus is passed. The sensor may be acamera which may be contained within the stylus or may otherwise beassociated with the stylus. The image which is sensed through the lighttransmissive tip of the stylus may be stored or immediately utilized toproduce an image which visually simulates the texture of the sensedsurface or image.

One embodiment utilizes a transparent tip on one end of a stylus as alens for a camera. An image sensor which may be a pinhole camera orother suitable image sensing device is contained within the stylus suchthat the image sensor detects the surface texture of the surface overwhich the clear tip of the stylus is passed. The sensed image is sent toa host computer and may be used to produce a newly created image whichincludes visual texturing simulating that of the sensed surface. Thestylus tip may be physically touched to, passed close to, or passed somedistance from the sensed surface depending upon the sensitivity of thecamera, the focal point of the tip lens and the desires of the user. Byvarying the level of detail of the image sensed by the camera, the usermay thus determine the level of detail of the type of textured surfacewhich may be captured by the device.

The stylus may include an activation switch or other suitable device toallow the user of the stylus to determine the time and place at whichthe sensed surface image may be captured. Illumination from an externalsource or from an illumination device such as a Light Emitting Diode(LED) inside the stylus may be provided to assist in the image capture.The tip of the stylus may function as a lens to present the image to theimage sensor device. In one embodiment, the tip of the stylus may belight transparent such that the image sensor device may not require anyadditional lens structure other than that which is provided by the tipof the stylus. In some embodiments, the illumination may vary to matchor symbolize (rather than match) a color or texture of a portion of acaptured image, a color or texture over which the stylus is positioned,a color or texture corresponding to an output of the stylus or an outputsignal generated by the stylus or an associated electronic device, andso on.

In another embodiment of the device, the sensed image is provided to adata storage device which may then store the image for future use. Thedata storage device could be contained within the stylus or in a laptopcomputer or electronic tablet or other suitable device which may storethe sensed texture image. The sensed image may then be utilized at alater time with the user's device or with a different device of the useror of another user. The stored image may then be communicated to theoriginal or to another stylus to use as a pen or brush in electronicpainting applications. The sensed image may be communicated to the datastorage device wirelessly or through a direct connection to the stylus.

In yet another embodiment of the device the physical feel of an object,an object in a sensed image, an object corresponding to a portion of thesensed image, or the sensed image itself, may be provided to a userthrough tactile sensor feedback from a haptic device. In thisembodiment, the user is able to experience both the look and feel of thesensed surface image or corresponding object. The visual appearance ofthe sensed surface/object is provided on a screen of the electronicdevice while the feel of the surface may be electronically provided tothe user through known haptic devices. Such feedback may also beprovided based on a color of a sensed image or object, as anotherexample.

Another embodiment of the device may be used to create three dimensionalimages. In this embodiment the texture surface may be a threedimensional representation of a sculpture or other physicallyrepresented surface such as a tool design. In this embodiment, thestylus may be used as part of a three dimensional computer aided design(CAD) system. For example, the stylus may be used to design a complexsurface required for a specific tooling application.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a perspective view of an electronic device held by a user;

FIG. 2 is a perspective view of an electronic device;

FIG. 3 is a perspective view of an electronic device with a stylus;

FIG. 4 is a perspective view of an embodiment of a stylus;

FIG. 5 is a cutaway view of another embodiment of a stylus;

FIG. 6 is a view of a cloth image;

FIG. 7 is a view of a relief sculpture image;

FIG. 8 is a perspective view of engraved jewelry and a stylus;

FIG. 9 is a cutaway view of another embodiment of a stylus including anIMU;

FIG. 10 is a perspective view of a three dimensional map includingsurface features and a stylus; and

FIG. 11 illustrates a user employing a stylus and haptic feedback.

DETAILED DESCRIPTION

Generally, embodiments described herein disclose systems and methods forsensing, storing and utilizing an image including a textured surface.Those embodiments may enable graphic designers, engineers, draftsmen,artists, or any other individual to improve the quality and authenticityof images generated electronically by including textured look and feelto those images.

Referring to the figures wherein like reference numbers denote likestructure throughout the specification, FIG. 1 is a perspective view ofan electronic device such as a tablet computer 11 is shown held by auser 12. Tablet computer 11 includes a display screen 13 to displayimages. FIG. 2 is a perspective view of tablet computer 11 and screen13. FIG. 3 is a perspective view of tablet computer 11 and screen 13with a stylus 14 shown contacting screen 13. Stylus 14 may be used inconjunction with tablet 11 and screen 13 to generate visual images onscreen 13 or to select certain features or applications from screen 13by touch. Computer tablet 11 can record video, take photos, play music,and perform Internet functions such as web-browsing and emailing. Otherfunctions of computer tablet 11, including playing games, GPSnavigation, social networking, and the like, can be enabled bydownloading and installing software applications, commonly known as“apps”.

Display 13 may include various sensors. For example an ambient lightsensor (not shown) to adjust screen brightness and a 3-axisaccelerometer (not shown) to sense orientation and switch betweenportrait and landscape modes. Tablet computer 11 may support orientationin four orientations, including upside-down. Tablet 11 may includecontrols to control the device 11 and the display screen 13. Computertablet 11 may be controlled by the fingers of user 12 applied to ormoved adjacent screen 13 (a touch screen) or as shown in FIG. 3 by astylus 14 which may itself be applied to or moved adjacent screen 13.Styli such as stylus 14 have been used to control various functions ofcomputer tablet 11 such as for example, in word processing applications,to play games, and as drawing tools for various applications.

Referring to FIG. 4, stylus 14 is illustrated which includes a lighttransmissive tip 15, and an off/on button 16. Referring to FIG. 5, aside sectional view of stylus 14 is shown. Tip 15 is configured to actas a lens for an image capture device such as a camera 17 or othersuitable image capture device. A power source 18 is electromagneticallyconnected to off/on button 16 and camera 17 such that user 12 may holdstylus 14 and turn camera 17 on or off by operating button 16. Camera 17may be a fine scale camera or other suitable imaging device which isused to sense the depth of the image over which tip 15 is passed. Apinhole camera with minimal lens requirement may be used as camera 17. Adata transmission device 19 is electromagnetically connected to camera17 and may transmit data to tablet 11 either wirelessly or by directconnection as is known in the art. A sensed image may then be displayedon screen 13 of computer tablet 11 to allow user 12 to observe the imageof the surface texture which is sensed by stylus 14. A light source 21to provide illumination for camera 17 is connected to power supply 18and a memory device 22 to store the sensed image data iselectromagnetically connected to camera 17. Both light source 21 andmemory 22 may be included within, or otherwise associated with, stylus14. It should be appreciated that any or all of the various elementsshown within or comprising the stylus may be electrically and/orphysically connected to one another.

Camera 17 may thus be employed to sense the surface over which tip 15 ispassed. That is, tip 15 may be passed some distance over, or may bedirectly in contact with, a surface, depending upon the focal length oflens 15 and the resolution of camera 17. Camera 17 will sense the depthsof feature variations on a surface and the change in those distances todetermine a texture of the surface and capture that image. It should beappreciated that certain embodiments may include a capture mode duringwhich such variations, textures, features and physical characteristics,and in some cases colors, may be sampled or otherwise captured. Further,embodiments may employ an output or drawing mode in which the capturedphysical characteristics are reproduced on a display of an associatedelectronic device in response to motion of the stylus or other inputfrom the stylus (or another input device entirely, thereby permittingthe stylus to be used solely for selection and capture).

Stylus 14 may be wirelessly or otherwise electromagnetically connectedto computer tablet 11 through transmission device 19 such that the imagecaptured by camera 17 may be transmitted to computer tablet 11 and thatimage may be displayed on screen 13. Because camera 17 captures thevariations in depth from the surface over which tip 15 is passed, theimage generated on screen 13 may include a textured appearance. That is,the variations in the surface are displayed in the form of an image withthe look and feel of the surface itself. So, for example, stylus tip 15may be passed over a cloth surface 23 such as that shown in FIG. 6 andthe image generated on screen 13 will reflect the surface variationssuch that user 12 is able to visually reproduce the surface texture ofcloth surface 23 on screen 13. This reproduction may occur with respectto any shape, line, or graphic created in response to action of thestylus, such as motion while in a drawing mode.

Likewise, the haptic feedback or feel of the stylus may simulate thefeel of touching the cloth surface 23 or any other surface, or of movinga stick or pen over the cloth surface. Thus, as the physicalcharacteristics of the surface captured by the stylus change, so too maythe generated image and the haptic feel that result from operation ofthe stylus. This may provide a user with a tactile indication of thegraphic being generated by operation of the stylus, or the graphic to begenerated through stylus operation. In some embodiments, the hapticfeedback may not simulate any physical characteristic of a surface butmay still vary as different surfaces are captured and/or correspondinggraphics generated in response to stylus operation. As another example,the haptic output of the stylus may correlate to a color rather than atexture or feel. As yet another example, the haptic output may correlateto a combination of a color and texture, such that a change to eithercolor or texture results in a change to the haptic output. In someembodiments, two haptic output signals may alternate to provide feedbackto the user (for example, one output for a color and one for a texture).In another, a single haptic output may be selected, generated orotherwise provided based on both color and texture/feel. For example, afirst waveform corresponding to a color output may be combined, merged,multiplexed, or the like with a second waveform corresponding to atexture and the resulting waveform may be used for haptic output of theparticular color/texture combination.

In order to provide uniformity in the sensed image, it may be desirableto sample the texture at various positions on the surface. Thus, forexample, sensing the surface characteristics at the endpoints and atvarious points in the center of the surface and then combining thoseimages into a composite image may be useful to prevent repetitivepatterns appearing in the reproduced image. The images sensed atdifferent locations on the surface such as cloth 23 may be combined toproduce a uniform textured surface. The same may be done with variouscolors across different regions of an image. Alternatively, the stylus14 could be held at different distances from the surface to be sensedwhich would provide varying perspectives on the texture. These imagesmay be stored in memory device 22 and then combined through the use ofcomputer programming.

Further, it should be appreciated that the haptic feedback produced bythe stylus, during its operation, may vary with the characteristics ofthe surface sensed or otherwise captured by the stylus. A surface orsurface image may be captured at a first distance, for example, andprovide a first haptic feel or feedback. That same surface may provide adifferent haptic feel or feedback for a user if the surface (or an imagethereof) si captured at a second distance, insofar as the field of viewof the camera may change, the relative size of any surface features ortextures may change within an image captured by the camera, and so on.

Still other embodiments may combine some form of distance sensing ormeasurement with the aforementioned camera in order to provide a uniformhaptic feel or feedback for any given surface, regardless of thedistance at which the surface is sampled or otherwise captured. This maypermit a stylus to compensate for image capture occurring at differingdistances, as one example.

Multiple samples at different locations on surface 23 can be taken toensure that repetitive images do not result in a visual image whichincludes the exact same features. The reproduced image may then berandomized and blended with various individual images such that thefinal composite image does not contain repetitive features. Thiscapability brings added dimensions and applications to computer tablet11 such as, for example, in art, computer design, and three dimensionalmodeling.

In order for camera 17 in stylus 14 to correctly sense the surface,illumination of the surface to be sensed is desirable. Thus, anillumination device 21, which may be a light emitting diode or othersuitable device may be associated with, or included in, stylus 14 toilluminate the surface such as cloth surface 23 in FIG. 6.Alternatively, external sources of illumination (not shown) may be usedto provide proper illumination of surface 23. In either case, theillumination of the surface to be sensed is sufficient to allow camera17 to obtain an image of the surface texture of cloth surface 23. Tocapture the textured surface of cloth 23, stylus 14 may be passed overvarious portions of cloth surface 23 and the images thus sensed bycamera 17 may be recorded and saved in a memory device as describedbelow.

Referring again to FIG. 5, stylus 14 is shown which includes a memorydevice 22. The inclusion of memory device 22 enables the storage ofsurface images sensed by camera 17. Thus, for example, a user 12 couldpass stylus tip 15 over a surface to detect a surface texture and thatimage stored in memory 22. That surface texture could then be used lateron that user's, or a different user's, computer tablet 11. This imagestorage feature may enable collaboration of users to produce aparticular visual image which incorporates the sensed surface textures.It also permits the user to incorporate various surface textures intoone visual image or to combine those textures to produce a differenttexture. Such combinations would be enabled by software applications incomputer tablet 11 as is known in the art. A stored image may be used ata later time or in a different device or application. While memorydevice 22 is shown as included in stylus 14, it can be appreciated thatmemory device 22 could be located apart from stylus 14, such as, forexample, in tablet 11.

Another embodiment allows the device to sense surface textures andfeatures such as those shown in FIG. 7, and transmit a haptic output toa user corresponding to such textures and/or features. That is, surfacetextures or variations in a relief painting 24 in FIG. 7 may then bevisually displayed on computer tablet 11 or stored as described above tolater produce a visual image which represents the various surfacecharacteristics of the sensed image in FIG. 7. Further, as the stylus ismoved to generate portions of the visual image, the stylus may alsoprovide haptic feedback corresponding to the texture, features, surfacecharacteristics, and/or other physical characteristics of the stylusoutput, or an image generated in response to stylus output.

Variations of an image could also be enabled by software applications incomputer tablet 11 as is known in the art. Various images of reliefpainting 24 may be captured by stylus 14 and then those images could becombined to reproduce an image, which may be a three dimensional image,of the relief painting 24 on electronic device 11. In a similar manner,by utilizing stylus 14, a user could also visually reproduce a threedimensional object such as a sculpture.

The ability for a user to sense surface texture variations adds a depthdimension to the visual images which may be generated. No longer will auser be limited by preprogrammed textures or surface features. A usercan generate his or her personalized surface textures. These texturesmay then be integrated or combined in order to produce additionaltextures in generated visual images. Faux painting or faux finishing areterms used to describe a decorative paint finishes that replicate theappearance of materials such as marble, wood or stone. Using stylus 14in these applications allows a user to generate realistic twodimensional images by using stylus to sense the texture of the actualsurfaces. By employing haptic feedback, designers and modelers mayreceive touch feedback relating to the “surface” they are sculpting orcreating, allowing faster and more natural workflow than traditionalmethods.

Another embodiment of the invention can be implemented to engravejewelry or other items such as metals, plastics, glass and crystal. Itis now commonplace for retail stores (mostly jewelry, silverware oraward stores) to have a small computer controlled engraving capabilityon site. This enables them to personalize the products they sell.Referring to FIG. 8, an engraved necklace 25 is shown. Engraving 26 maytake any decorative or other aesthetically pleasing form as is known inthe art. Engraving 26 could take the form of an inscription conveyingsentiments or other information. Retail engraving machines tend to befocused around ease of use for the operator and the ability to engrave awide variety of items including flat metal plates, jewelry of differentshapes and sizes, as well as cylindrical items such as mugs andtankards. They will typically be equipped with a computer dedicated tographic design that will enable the operator to easily design a text orpicture graphic which the software will translate into digital signalstelling the engraver machine what to do. The present invention would beemployed to assist in this process. The user would pass the stylus 14over a surface such as engraving 26 in FIG. 8 and the sensed texturecould then be used by the engraving machine to reproduce that sensedtextured surface on another item. In this way, copies of engravings,signatures etc. could be reproduced or custom engravings etc. could beproduced by combining various sensed texture surfaces.

An inertial measurement unit (IMU) is used in many navigation systemssuch as for aircraft, watercraft and others form of transportation. Thedata collected from the IMU's sensors allows a computer to track theposition using a method known in the art as dead reckoning. IMU's detecta current rate of acceleration using one or more accelerometers anddetect changes in rotational attributes like pitch, roll and yaw usingone or more gyroscopes. Some IMUs include a magnetometer to assist incalibration against orientation drift. In an alternate embodiment,referring to FIG. 9, stylus 14 includes an IMU 32 in addition to camera17, power source 18, data transmission device 19, light source 21, andmemory device 22. Including an IMU allows the stylus 14 to be used togenerate three dimensional images as discussed above. It should beappreciated that any or all of the various elements shown within orcomprising the stylus may be electrically and/or physically connected toone another.

An IMU 32 could be used in one embodiment in conjunction with camera 17to generate a three dimensional image from a two dimensional image. Forexample, stylus 14 could be used to scan an image on screen 13 and, inconjunction with IMU 32, to produce a three dimensional image. With theinclusion of IMU 32, stylus 14 could be used to generate precisemeasurements as is known in arts such a plotting and vector graphics.Vector graphics is the use of geometrical primitives such as points,lines, curves, and shapes or polygons, which are all based onmathematical expressions, to represent images in computer graphics.Vector graphics are based on vectors (also called paths, or strokes)which lead through locations called control points. Each of these pointshas a definite position on the x and y axes. Each point, may include thelocation of the point in the work space and the direction of the vector(which is what defines the direction of the track). Each track can beassigned a color, a shape, a thickness and also a fill. By including anIMU 32 in stylus 14, the precise location of stylus location as well asthe orientation of stylus 14 with respect to a reference point can bedetermined by IMU 32. With this information a three dimensional imagemay be reproduced from a two dimensional image on screen 13 or fromanother two dimensional image on a different device or in a differentmedium as is known in the computer graphics art.

Another embodiment of the invention would allow computer aided design(CAD) engineers to utilize the stylus 14 to produce images which includesurface texture as part of the image or to produce three dimensionalimages. CAD uses computer systems to assist in the creation,modification, analysis, or optimization of a design. CAD software isused to increase the productivity of the designer, improve the qualityof design, improve communications through documentation, and to create adatabase for manufacturing. CAD output may be in the form of electronicfiles for print, machining, or other manufacturing operations. CAD isalso widely used to produce computer animation for special effects inmovies, advertising and technical manuals, often called digital contentcreation. In one embodiment, texture images captured by stylus 14 may beutilized in digital content creation. That is, texture images stored inmemory 22 may be sent by transmission device 19 to a computer aideddesign system which may then implement the texture images in a digitalcontent creation application.

CAD may be used in designing electronic systems, mechanical design orcomputer-aided drafting which includes the process of creating atechnical drawing with the use of computer software. CAD software formechanical design uses either vector-based graphics to depict theobjects of traditional drafting, or may also produce raster graphicsshowing the overall appearance of designed objects. CAD may be used todesign curves and figures in two-dimensional (2D) space; or curves,surfaces, and solids in three-dimensional (3D) space. Stylus 14 could beused in the implementation of such design, particularly in the threedimensional design applications.

Other industrial design applications such as for machining of metalparts or other industrial uses such as engraving stone, clay or similarmaterials become apparent when the texture images captured by stylus 14are utilized. In addition, as discussed above, haptics, when used incombination with the stylus of the present invention in virtual orvisual arts, such as graphic design, allow the artist to have directcontact with a virtual instrument that produces real-time images. Thisembodiment provides such designers with more tools to produce moreinnovative and useful visual images and designs.

In yet another embodiment, stylus 14 could be used to sense variationsin surface texture in three dimensional mapping applications. Mapping iscreating graphic representations of information using spatialrelationships within the graphic to represent some relationships withinthe data. One type of mapping is the scaled portrayal of geographicalfeatures, that is, cartography. In geographical maps the images cancorrelate directly to geo-spacial properties, or they may symbolizeabstractions such as political borders. Traditionally, the paper (orplane) represents the surface of the earth. In this embodiment, stylus14 is passed over the surface to be mapped and the sensed image is thenused to reproduce a visual image which displays various surfacecharacteristics. In a geographic mapping application, this could be usedto generate a visual image which provides topographic information. Thisembodiment could be useful in surveying as well as map creation. Forexample, in FIG. 10, stylus 14 may be passed over three dimensional map27 and images of surface variations or features, such as mountain ranges28 could be captured and sent to computer tablet 11 in order to create avisual two dimensional image of the captured three dimensional image 27thus providing topographic detail to the two dimensional image.

In addition to producing visual images, the invention could employhaptics to provided tactile feedback to a user. Haptic technology, orhaptics, is a tactile feedback technology which takes advantage of thesense of touch by generating forces, vibrations, or motions to the user.This mechanical stimulation can be used to assist in the creation ofvirtual objects in a computer simulation, to control such virtualobjects, and to enhance the remote control of machines and devices(telerobotics). Haptic devices may incorporate tactile sensors thatmeasure forces exerted by the user on the interface. Thus, in oneembodiment of the present invention, stylus 14 may be used by a clothingmanufacturer to sense the texture of, for example, cloth 23 in FIG. 6.The texture of the cloth may be extracted from a capture image of thecloth, as one example. As the stylus is moved over the cloth (or astationary image of the cloth is captured), image processing may beemployed to extract a texture or other physical feature of the cloth.

The sensed texture surface would then be stored as part of the visualcloth image. The cloth image would then be put on the manufacturer'swebsite. A user would then pass his or her stylus 14 over the image andhaptic feedback devices could be used to reproduce the sensed texture ofthe visual image to a user employing the haptic feedback device. Theuser would thus sense the texture of the cloth. As an example of oneapplication of this embodiment in the retail clothing industry, userswould utilize the sensed texture through haptic feedback and be able to“feel” the texture of clothes for sale on the internet. Likewise, suchfeedback may be provided based on a color of a sensed image or object,and/or a combination of color and texture, as previously described.

For example, referring to FIG. 11, in another embodiment a hapticfeedback device which may be, for example, a rumble pack 29 such as isknown in the electronic gaming arts could be used in conjunction withstylus 14 to generate haptic feedback to a user. An image, such astexture image associated with cloth 23 in FIG. 6, is captured by stylus14 as discussed above. That captured image is displayed on screen 13 oftablet 11. Stylus 14 is held by user 12 in one hand while rumble pack 29is simultaneously held in another hand 31 of user 12 as shown in FIG.10. As stylus 14 is passed over screen 13, the image is sensed by stylus14 and the corresponding texture information is conveyed to rumble pack29 by software associated with computer tablet 11. User 12 is thus ableto visually observe the image on screen 13 while feeling the texture ofthe displayed image through rumble pack 29. The combination of visualand tactile input to user 12 allows the user to experience the “look andfeel” of a fabric in the example of FIG. 6. Other haptic feedbackdevices as are known in the art could also be employed in conjunctionwith stylus 14 to generate tactile feedback to a user. It should beappreciated that substantially any suitable haptic feedback mechanismmay be employed in embodiments.

In an alternate embodiment, a visually impaired person could use stylus14 and rumble pack 29 as described above. That embodiment would allowthe visually impaired person to “feel” an image which is displayed onscreen 13 that he or she is otherwise unable to see due to theimpairment. The combination of two senses, visual and touch, makes thisembodiment useful in other applications such as medicine where a medicalprofessional may use this application to “feel” the image of an x-ray ora magnetic resonance image (MRI).

In certain embodiments, a display may be provided on the stylus, or maybe associated with the stylus, and may show a texture and/or color of anoutput from the stylus. Thus, if moving the stylus produces an outputsignal that draws a line having a particular texture and/or color, thatsame texture and/or color may be shown on the display even if the stylusis immobile. Accordingly, a user may relatively easily determine whatthe output of the stylus is or will be without needing to activate ormove the stylus.

Generally, embodiments have been described herein with respect tocapturing a texture or other physical characteristic of a surface, andoutputting that texture/physical characteristic during or as anoperation of the stylus, as well as providing haptic feedback or otherfeel that simulates or varies with the texture/physical characteristic.However, it should be appreciated that certain visual characteristics,such as color, brightness, saturation, and the like, may also be used asthe basis for providing certain haptic outputs to a user. Thus, forexample, the haptic output of a stylus during a selection operation or adrawing operation may vary depending on the color and/or brightness ofthe generated graphic.

While the disclosure has described various embodiments, it should beexpressly understood to those of ordinary skill in the art that certainmodifications may be made without departing from the spirit or scope ofthis disclosure. For example, while various configurations have beendisclosed for a stylus to enable various applications for texturecapture, additional capabilities may be employed without departing fromthe spirit or scope of the disclosure. Accordingly, the proper scope ofthis disclosure is set forth in the following claims.

We claim:
 1. A stylus comprising: a housing, a tip portion on saidhousing to transmit an image of a surface adjacent to said tip portion;an image sensing device associated with said tip portion to capture saidimage; and a storage device electromagnetically connected to said imagesensing device.
 2. The stylus of claim 1, wherein the captured imagecomprises a texture.
 3. The stylus according to claim 1 wherein saidimage sensing device includes a camera.
 4. The stylus according to claim1 wherein said tip portion includes a light transmissive portion.
 5. Thestylus according to claim 1 further including a light source associatedwith said image sensing device.
 6. The stylus according to claim 5wherein said light source includes a light emitting diode in saidhousing.
 7. The stylus of claim 1 wherein said image sensing device isoperative to capture a texture of the image and a color of the image. 8.The stylus of claim 1 wherein said surface includes variations in adistance of said surface from said tip portion.
 9. The stylus accordingto claim 1 further including a device for supplying power to saidstylus.
 10. The stylus according to claim 1 further including aninertial measurement unit (IMU).
 11. A system comprising: an electronicdevice; a display screen associated with said electronic device; astylus electromagnetically connected to said electronic device, saidstylus including: a tip portion at one end of said stylus; an imagesensing device associated with said tip portion to capture an image of asurface adjacent to said tip portion; a transmitting deviceelectromagnetically connected to said image sensing device; and wherebysaid image is displayed on said screen.
 12. The system of claim 11wherein said surface includes variations in the distance of said surfacefrom said tip portion.
 13. The system of claim 11 wherein said imagesensing device includes a camera.
 14. The system of claim 11 whereinsaid tip portion includes a light transmissive portion.
 15. The systemof claim 11 wherein said image comprises a texture.
 16. The system ofclaim 15 wherein said texture corresponds to said surface.
 17. Thesystem of claim 16 further including an inertial measurement unit (IMU).18. The system of claim 16 further including a haptic feedback device,the haptic feedback device operative to output a haptic feedbackcorrelated to said texture.
 19. A method for displaying an image of asurface on an electronic device comprising the steps of: scanning astylus over said surface; sensing an image of said surface in saidstylus; capturing said image in said stylus; and transmitting said imageto said electronic device.
 20. The method of claim 19 wherein saidsurface includes variations in the distance of said surface from saidstylus.
 21. The method of claim 19 further including the step ofextracting a texture from said image.
 22. The method of claim 19 whereinthe step of scanning includes sensing said image at multiple locationson said surface.